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Abstract 


The  problem  of  rendering  the  origin  an  asymptotically  stable  equilibrium  point  of  a  nonlinear  system 
while,  at  the  same  time,  optimizing  some  measure  of  performance  has  been  the  object  of  much  attention 
in  the  past  few  years.  In  contrast  to  the  case  of  linear  systems  where  several  optimal  synthesis  techniques 
(such  as  9-Ly  and  are  well  established,  their  nonlinear  counterparts  are  just  starting  to  emerge. 
Moreover,  in  most  cases  these  tools  lead  to  partial  differential  equations  that  are  difficult  to  solve.  In  this 
chapter  we  propose  a  suboptimal  regulator  for  nonlinear  parameter  varying,  control  affine  systems  based 
upon  the  combination  of  model  predictive  and  control  Lyapunov  function  techniques.  The  main  result  of 
the  chapter  shows  that  this  controller  is  nearly  optimal  provided  that  a  certain  finite  horizon  problem  can 
be  solved  on-line.  Additional  results  include  (a)  sufficient  conditions  guaranteeing  closed  loop  stability 
even  in  cases  where  there  is  not  enough  computational  power  available  to  solve  this  optimization  on-line; 
and  (b)  an  analysis  of  the  suboptimality  level  of  the  proposed  method. 


*This  work  was  supported  in  part  by  NSF  grants  ECS-9625920  and  ECS-9907051  and  AFOSR  under  grant  F49620-00-1- 
0020. 


1  Introduction 


A  large  number  of  control  problems  involve  designing  a  controller  capable  of  rendering  some  point  an 
asymptotically  stable  equilibrium  point  of  a  given  time  invariant  system  while,  simultaneously,  optimizing 
some  performance  index.  In  the  case  of  linear  dynamics  this  problem  has  been  thoroughly  explored  during 
the  past  decade,  leading  to  powerful  formalisms  such  as  /r-synthesis  and  t  optimal  control  theoiy  that  have 
been  successfully  employed  to  solve  some  hard  practical  problems. 


In  the  case  of  nonlinear  dynamics,  popular  design  techniques  include  Jacobian  linearization  (JL)  [14], 
feedback  linearization  (FL)  [14],  the  use  of  control  Lyapunov  functions  (CLF)[1, 24]  and  recursive  backstep- 
ping  [14].  While  these  methods  provide  powerful  tools  for  designing  stabilizing  controllers,  performance 
of  the  resulting  closed  loop  systems  can  vary  widely,  as  we  illustrate  in  the  sequel  with  the  problem  of  con¬ 
trolling  a  thrust  vectored  aircraft.  A  simplified  planar  model  of  the  system  is  shown  in  Figure  1,  with  the 
corresponding  dynamics  given  by  (see  [30,  7]  for  details): 


X  —gsinQ 

y  =  g(co50-l)  -t- 

_ej  [  0  _ 


(1) 


where  x,  y  and  6  denote  horizontal,  vertical  and  angular  position  respectively  and  where  ui  and  U2  =  U2+Tng^ 
are  the  control  inputs. 
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Figure  1:  Simplified  model  of  a  thrust  vectored  aircraft 

Assume  that  the  goal  is  to  drive  the  system  to  the  origin,  while  minimizing  a  performance  index  of  the 
form 


J{xo,u)  =  J  [^'Q^-\-u'Ru]dt, 

0 

4(0)  =  [0  0  0  12.5  0  0] 

Q  =  diag[5  5  111  5],/?  =  72x2 


corresponding  to  the  following  choice  of  state  variables:  4  =  [•«  y  6  i  y  6]. 

‘Following  [30]  the  control  U2  has  been  shifted  to  compensate  for  gravity. 


(2) 

(3) 

(4) 
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method 

cost 

exact 

1115 

CLF  [7] 

2.53  X  10^ 

JL  +  LQR  [7] 

1.1  X  10^ 

Table  1 :  Comparison  of  different  methods  for  the  thrust  vectored  aircraft  example 


Table  1  compares  the  performance  achieved  by  several  commonly  used  nonlinear  control  design  meth¬ 
ods.  As  shown  there,  in  this  case  the  performance  of  the  CLF  and  JL  controllers  is  worse,  by  an  order  of 
magnitude,  than  the  optimal  cost  (obtained  by  offline  optimization  using  a  conjugate  gradient  algorithm). 
Indeed,  a  recent  workshop  on  nonlinear  control  [7]  has  shown  that  while  the  methods  mentioned  aboved  can 
recover  the  optimal  control  under  certain  conditions,  in  general  there  are  no  guarantees  on  the  performance 
of  the  resulting  system. 

As  an  alternative,  nonlinear  counterparts  of  [3,  11]  and  t  [15]  have  recently  started  to  emerge. 
While  these  theories  can  guarantee  optimality  (at  least  in  a  certain  sense),  from  a  practical  standpoint  they 
suffer  from  the  fact  that  they  lead  to  Hamilton-Jacobi-Isaacs  type  partial-differential  equations  that  are  hard 
to  solve,  except  in  some  restrictive,  low-dimensional  cases. 

Given  these  practical  difficulties,  during  the  past  few  years  there  has  been  an  increased  interest  in  ex¬ 
tending  receding  horizon  (RH)  techniques  to  nonlinear  plants.  These  techniques  are  appealing  since  they 
allow  for  explicitly  handling  constraints  and  guarantee  optimality  in  some  sense.  Moreover,  since  the  opti¬ 
mization  is  carried  only  along  the  present  trajectory  of  the  system  (i.e  “locally”)  the  resulting  computational 
complexity  is  far  less  than  that  associated  with  finding  the  true  global  optimal  control  (a  task  that  entails 
solving  a  Hamilton-Jacobi  type  equation).  However,  in  contrast  with  the  linear  case  (where  global  stability 
has  been  established  [27,  23,  21]),  for  nonlinear  plants  only  local  stability  results  are  available  [17].  Sev¬ 
eral  modified  nonlinear  RH  formulations  addressing  this  problem  have  been  proposed,  mostly  based  on  the 
use  of  additional  constraints  or  a  terminal  penalty.  For  instance,  [18]  uses  dual  control,  where  an  explicit 
optimization  is  used  to  drive  the  system  to  a  neighborhood  of  the  origin,  where  a  locally  stabilizing  linear 
control  law  is  used.  Magni  [16]  uses  a  terminal  penalty  obtained  assuming  that  a  linear  control  law  will 
be  used  after  the  optimization  horizon  T.  Finally,  [20]  achieves  stability  by  enforcing  an  additional  state 
constraint.  However,  while  these  approaches  guarantee  closed-loop  stability,  they  may  do  so  at  the  expense 
of  performance. 

In  this  chapter  we  propose  an  alternative  controller  for  suboptimal  regulation  of  nonlinear,  parameter 
varying,  control  affine  systems,  based  upon  the  combination  of  receding  horizon  and  control  Lyapunov 
functions  ideas.  This  approach  follows  in  the  spirit  of  a  similar  controller  successfully  used  in  the  case  of 
constrained  linear  systems  [25,  26,  27],  where  receding  horizon  was  used  to  drive  the  system  to  an  invariant 
neighborhood  of  the  origin  where  a  stabilizing  controller  is  available.  In  the  first  part  of  the  chapter  we  show 
that  combining  these  ideas  with  a  suitable  finite-horizon  approximation  of  the  performance  index,  leads  to 
globally  stabilizing,  nearly  optimal  controllers,  provided  that  enough  computational  power  is  available  to 
solve  on-line  an  optimization  problem.  In  the  second  part  of  the  chapter  we  show  how  to  modify  these 
controllers  to  guarantee  global  stability  in  the  face  of  computational  time  constraints.  Additional  results 
include  an  analysis  of  the  suboptimality  of  the  proposed  method  and  show  that  if  an  approximate  solution  to 
the  problem  is  known  in  a  set  containing  the  origin,  then  our  controller  yields  an  extension  of  this  solution 
with  the  same  suboptimality  level.  Finally,  we  show  that  in  the  limit  as  the  optimization  horizon  T  ^  0,  the 
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method  reduces  to  the  well  known  inverse  optimal  controller  of  Freeman  and  Kokotovic  [8]. 


2  Preliminaries 

2.1  Notation  and  Definitions 


In  the  sequel  we  consider  the  following  class  of  control-affine  nonlinear  parameter  varying  (NLPV)  systems: 


i  =  /[j:,p(t)]  +  g[^,p(0]M  (5) 

where  £  /?"  and  «  £  R'"  represent  the  state  and  control  variables,  the  vector  fields./(., .)  and  g(., .)  are 
known  C'  functions,  and  where  p  £  R"p  denotes  a  vector  of  time-varying  parameters,  unknown  a  priori,  but 
available  to  the  controller  in  real  time.  Further,  we  will  assume  that  at  all  times  p(r)  £  fP  C  R^o,  where  fP  is 
a  given  compact  set,  and  that  the  set  of  admissible  parameter  trajectories  is  given  by: 

Jv  =  {p  e  d(R,R''») :  p(0  £  !P,y,-  <  p,(r)  <  v,-,/=  l,2,...np,Vt  £  /?+}  (6) 

where  Vj  and  v,  are  given  numbers. 


Definition  1  A  positive  definite  function  V  :  R"  x  R  ^  R+ is  a  Control  Lyapunov  function  (CLF)for  the 
system  (5)  if  it  is  radially  unbounded  in  x  and 


r  dV  ' 

inf  LfV{x,p)  +  LgV{x,p)u+^p 


<  -a(.r)  <0,  V  jf  0  and  all  p€  % 


(7) 


where  a(.)  is  a  positive  definite  function,  and  where  LhV{x,p)  =  ^/i(jc,p)  denotes  the  Lie  derivative  ofV 
along  h. 


2.2  The  Quadratic  Regulator  Problem  for  NLPV  Systems 

i 

Consider  the  NLPV  system  (5).  In  the  sequel  we  consider  the  following  problem:  Given  an  initial  con¬ 
dition  Xo  and  an  initial  value  of  the  parameter  Po,  find  a  parameter  dependent  state-feedback  control  law 
M[x(r),  p(r)]  that  minimizes  the  following  performance  index: 


J{xo,po,u)=  sup  [  [x!Q{x,p)x+u'R{x,p)u]dt,x{(i)=Xo  (8) 

P€  Jv,P(0)=p„  7 

where  Q{.)  and  /?(.)  are  C*,  positive  definite  matrices^.  In  the  sequel,  for  simplicity,  the  explicit  dependence 
of  matrices  on  x  and  p  wilt  be  omitted,  when  it  is  clear  from  the  context. 

^This  condition  can  be  relaxed  to  Q{x,  p)  >  0 
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By  using  Pontryagin’s  principle  it  can  be  shown  that  solving  this  problem  is  equivalent  to  solving  the 
following  Hamilton-Jacobi-Bellman  type  partial  differential  equation: 

0  “  4  ^  V  <l^V( ^ (0>  P)  =  0  (9) 

If  this  equation  admits  a  C*  nonnegative  solution  V^,  then  the  optimal  control  is  given  by  u(x,  p)  =  -  's'w 
and  V(;c,p)  is  the  corresponding  optimal  cost  (or  storage  function),  i.e. 

oo 

V (a:,  p)  =  min  sup  /  (a/Qx+u'Ru)  dt 


3  An  Equivalent  Finite  Horizon  Regulation  Problem 


Unfortunately,  the  complexity  of  equation  (9)  prevents  its  solution,  except  in  some  very  simple,  low  dimen¬ 
sional  cases.  To  solve  this  difficulty,  motivated  by  the  work  in  [25,  28],  in  this  section  we  introduce  a  finite 
horizon  approximation  to  the  nonlinear  regulation  problem. 


Lemma  1  Consider  a  compact  set  S  containing  the  origin  in  its  interior  and  assume  that  the  optimal  storage 
junction  V  (x,  p)  is  known  for  allxeS,p^iP.  Letv  =  min;tg95  minpej.  V  (x,  p)  where  dS  denotes  the  boundary 

of  S.  Finally,  define  the  set  Sy  =  |x:  supp^j,  V  (x,  p)  <  v|.  Consider  the  following  two  optimization  problems: 

min  sup  <!  /(xo,  u,  p^)  =  J  [x!Qx-\-u'Ru]dT  \  (10) 

“  P65v,P(f)=P<-  *•  ‘  J 

min  sup  Wt(jco,u,Po)  =  f[x'Qx+u'Ru]dx+V[x(T),p(T)]]  (11) 

“  pe^v,p(0=p»  ^  f  J 

subject  to  ( 5)  with  x{t)  =  Xg.  Then  an  optimal  solution  of  problem  (11)  is  also  optimal  for  (10)  in  the  interval 
[f,  T]  provided  that  x[T)  £  Sy. 


Proof:  If  Xo  £  Sy  the  proof  follows  immediately  from  the  facts  that,  for  all  admissible  parameter  trajectories, 
Sy  is  positively  invariant  and  that  V(x,p)  is  the  optimal  return  function  there.  If  Xg  ^  Sy,  consider  the 
following  free  terminal  time  problem: 

r{x,p,t)  =  minsup  j  V[x(//),p(r/)]H-//^[x'QxH-M'/?H 

“  P67v  '• 
subject  to: 

X{tf)  £  Sy 

Let  x°,  m",  p"  denote  the  optimal  trajectory.  It  can  be  easily  seen  that  the  optimal  return  function  satisfies: 


^  dJ  dJ  .  IdJ  ,  ,dj'  .  dJ 

0  =  ■^-l-^/(x,p)- ^g  ^  -\-)(!Qx+  max  v,- 

at  dx  '  '  4dx  dx  yj<Vi<v,"dp, 


(13) 


with  boundary  condition  J[x,  p,  r)  =  V(x,  p)  for  x  £  Sy.  Clearly  this  equation  admits  as  solution  /(x,  p,  r)  = 
V(x,p).  Thus  problems  (10)  and  (12)  are  equivalent.  To  establish  the  claim  we  will  show  that  an  optimal 
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solution  of  (1 1)  is  also  optimal  for  (12)  (and  thus  (10)),  provided  that  xP{T)  G  5y.  To  this  effect  note  that 
the  Euler-Lagrange  optimality  conditions  for  problems  (11)  and  (12)  are  identical,  except  for  the  additional 
transversality  condition  H[u°,:)(F{tf),'k°{tf),jf_{tfj\  —  0  that  appears  in  the  latter,  where  X(r)  and  /i(r)  denote 
the  co-states  associated  with  the  states  and  parameters,  respectively.  The  boundary  conditions  for  these 
co-states  in  problem  (1 1)  are  given  by 


dV  dV 

Since  x{T)  G  Sy  it  follows  that  xf^{T),u?{T),X°[T),iJ°{T)  satisfy  the  HJB  equation  (9),  or  equivalently 
H[u° =0.  Thus,  an  optimal  solution  of  (11)  is  also  optimal  for  (12). 


□ 

This  lemma  shows  that  if  a  solution  to  the  HJB  equation  (9)  is  known  in  a  neighborhood  of  the  origin, 
then  it  can  be  extended  via  an  explicit  finite  horizon  optimization,  well  suited  for  an  on-line  implementation. 
This  suggests  the  following  RH  type  control  law: 


Algorithm  1 

0.  -  Data:  The  region  Sy,  the  function  V  (x,  p)  for  all  x  £  S^,  a  sampling  interval  8T. 

1-  lfx{t)  £S,.u  =  -i7?-‘g(x,p)'^' 

2.-  Ifx[t)  ^  5y  then  solve  a  sequence  of  optimization  problems  of  the  form  (11)  with  increasing  values  of 
T  until  a  solution  such  that  x{T)  £  Sy  is  found.  Use  the  corresponding  control  law  u{t)  in  the  interval 
[r,/  +  5r]. 


From  the  results  above  it  is  clear  that  the  resulting  control  law  is  globally  optimal  and  thus  globally  stabiliz¬ 
ing.  However,  the  computational  complexity  associated  with  finding  V (x,  p)  (even  only  in  the  region  Sy)  may 
preclude  the  use  of  this  control  law  in  many  practical  cases.  Thus,  it  is  of  interest  to  consider  a  control  law 
where  an  approximation  ^(x,  p)  (rather  than  V(x,  p))is  used.  To  this  effect  consider  a  compact  set  S  contain¬ 
ing  the  origin  in  its  interior  and  let  T:  S  x  fP  ->•  /?+,  W  £  x  R"p,R)  be  a  Control  Lyapunov  Function 

for  system  (5).  Finally,  let  c  =  min^teas  minpgj.  ^(x,  p)  and  define  the  set  S»f  C  S  =  {x:  maxpgy  ^(x,  p)  <  c}. 
Consider  the  modified  control  law: 


Algorithm  2 


0.-  Data:  a  CLF^{x,  p),  the  region  S^i,  a  sampling  interval  57,  a  positive  definite  function  (t(.). 
1.-  //x(r)eSv,;,  M^(x,p)=argmin  sup  {||M||:^[/(jf,p)+5(-«,p)M]  +  X,  gv,- < -ct(x)  <  oj 

U  y,<v,<Vi  '■  J 


2.-  If  X  ^  S^i  then  consider  an  increasing  sequence  f.  Let 


ux.  —  argmm  <  sup 
“  [_P€jv  I 


'  {x'Qx  +  u'Ru)dx  +  W[x{Ti),p{Ti)] 
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Denote  by  x*(.)  the  corresponding  optimal  trajectory  and  define?: 

J{xo,  Po,  Ti)  =  sup  \fp  {x*'Qx*  +  u*'Ru*)  (7^) ,  p(7^)]l 

7’ (jc,  Po)  =  argmiiiT-  {J(x,  po,  Ti) :  x*{Ti)  e  S^>  for  all  p(.)  6  p(0  =  Po} 

Then  wvp(;c,p)  =  “rwW'  ^  [?,f  +  5r]. 


Theorem  1  Assume  that  Q{x,  p)  >  Cml  >  0.  Then  the  control  law  u^/  generated  by  Algorithm  2  renders  the 
origin  a  globally  asymptotically  stable  equilibrium  point  of  (5). 


Proof:  Consider  first  an  initial  condition  Xg^S'^y,  an  initial  value  of  the  parameter  Po,  and  the  corresponding 
optimal  control  law  «*(.)  and  trajectory  Let 

%  f  tF{Xoypo)  ^ 

K^O,  Po)  =  sup  '  <  {x*'Qx*  +  u*'Ru*)  ^/x+ T {A:*[r(xo, Po)], p[7’(jCo. Po)]}  > 

P€%:p(l)=Po  J 


and  define  xi  =  x*{t  +  dt)  (with  dt  >  0  small  enough  so  that  x*(t  +  dt)  ^  S^>)  and  pi  =  p{t  +  dt),  where 
p(.)  denotes  any  admissible  parameter  trajectory  starting  at  Po-  Since  x*(t),  t  6  \t-\-dt,  T\  is  also  a  feasible 
trajectory  starting  from  x\  we  have  that: 


y(xi,pi) 


inf  sup  |/4£'‘^'^(x'ex+M'7?M)</T+'F{x[r(xi,pi)],p[7’(xi,pi)]}j 

“  pe5v:p(f+*)=Pi 


<  sup  /45j;’‘’"^(^*'(2^*+M*'^«O^T^+'*'{^*[7’(^«.po)].p[n'^o,Po)]} 

p6:rv:p(f+*)=pi 


< 


sup 

P6?'v:p(r)=p„ 


T(,Xo,Po) 


(x*'Qx*  +  u*'Ru*)  dz+W {x*  [T(xo ,  Po)] ,  p[r(x„,  p„)]} } 


(16) 


-  +  u*'Ru*)dz  =  J{xo)  -  Sl^‘^‘{x*'Qx^  +  u*'Ru*)dz 

Thus,  for  any  admissible  parameter  trajectory, 


j  ^  J[x{t  +  dt),p{t+dt)]-J [x(r) , p(r)] 

dt^O  dt 


<  -x{t)'Qx{t)  <  -Cm  ■  ||x||  <  0 


Since  /.(x,  p)  >  0  and  J(x,  p)  <  0  for  all  x  ^  5h<,  it  follows  that  all  trajectories  reach  the  set  S,;,  in  finite  time. 
Asymptotic  stability  now  follows  from  the  facts  that  Sv  is  invariant  with  respect  to  (i.e.  trajectories 
starting  in  the  set  never  leave  it)  and  that  'P(x,  p)  is  a  CLF  there. 


□ 


4  A  Modified  Receding  Horizon  Controller 


In  the  last  section  we  outlined  a  receding  horizon  type  law,  that  under  certain  conditions  globally  stabilizes 
system  (5).  While  most  of  these  conditions  are  rather  mild  (essentially  equivalent  to  the  existence  of  a  CLF), 

^From  Lasalle-Yoshizawa's Theorem  [14]  we  have  that  Wroj-^^xlQx  =  0.  Hence  r(A:<„po)  is  finite. 
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the  requirement  that  T  should  be  large  enough  so  that  x{T)  6  could  pose  a  problem,  specially  in  cases 
where  the  system  has  fast  dynamics.  Thus,  it  is  of  interest  to  consider  the  following  modified  control  law 
where  both  an  approximation  p)  (rather  than  V (a:,  p))  and  a  fixed  horizon  T  are  used: 


Algorithm  3 


0.-  Data:  a  CLF  'F(x:),  an  invariant  region  such  that  0  G  ^  horizon  T. 

//‘j:(r)  e  54*,  mh/(j:,p)  =  argmin  i  ||m||:  sup  Z^/'F+L^'Tw  +  Eg^v,- < -a(.r)  <  0  > 

u  [  Vi<vi<Vi  i  J 

2.-  Ifx{t)  ^  then  u^l{x,  p)  =  u{t)  where  u(k),  X  6  +  T]  is  given  by: 

’  T+t 

«  =  a^minsup<  j  {x!Qx+u'Ru)dx+'^[x{T +  t),p{T +  t)] 
u  J 


(17) 


subject  to: 


0 


>  ji:(r  +  7’)'(2x(r  +  7’)  +  min  sup  |v'/?v+^  |-M'(r)i?M 

^  A.t+T)\ 


(0 


(18) 


Note  that  inside  the  region  S^i,  the  control  action  is  a  generalization  to  the  NLPV  case  of  the  pointwise- 
minimum  norm  controller  proposed  in  [8].  Proceeding  as  in  [10]  it  can  be  shown  that  if  the  CLF  T'  has  the 
same  level  sets  as  the  optimal  value  function  V  inside  then  u^  is  indeed  the  optimal  solution  for  the 
original  problem.  Thus,  the  algorithm  allows  for  “switching-off’  the  on-line  optimization  when  inside  a 
region  where  the  optimal  storage  function  (or  a  good  approximation)  is  known. 


Theorem  2  The  control  law  u^  generated  by  Algorithm  3  has  the  following  properties: 

1.  It  renders  the  origin  a  globally  asymptotically  stable  equilibrium  point  of  (5). 

2.  It  coincides  with  the  globally  optimal  control  law  when  'F(x,  p)  =  V(j:,  p),  where  V (x,  p)  denotes  the 
optimal  storage  function  obtained  by  solving  (9). 


Proof:  To  prove  stability,  proceeding  as  in  Theorem  1,  consider  first  an  initial  condition  Xo  ^  S'y.  Denote  by 
u*,x*  the  optimal  control  and  associated  trajectory  respectively.  Then 

f  T+l+dt 

J[x{t  +  dt),p{t  +  dt)]  =  minsup<  /  {x'Qx+ufRu)dt  +  W[x{T  +  t  +  dt),p{T  +  t  +  dt)] 

“pi  t+dt 


T+t 

<  sup  /  (jc*'Qa:* +  «*'/?«*)  i/r  +  'P[x*(r  +  r),p(r+r)] 

P 

+  mmsup{x*'(7’  +  r)(2.^*(7’  +  r) +  v'/?v+ Y[A:*(r  +  r),p]}  Jr 


=  7Wr),p(r)]-  [.r‘(r)'Gx*(r)  +  «*'(r)^M*(r)]  Jr 
+  minsup{ji:’'(r  +  t)Qx*{T  +  r) +  v'7?v+T[x:*(7  +  r),p]}  Jr 

''  P 
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Therefore,  if  (18)  holds  then  we  have  that,  for  all  admissible  parameter  trajectories: 

J  =  +  ^0 . p(^  +^^0]  - ^[-^(0.  p(0]  <  _ct^||;,||2  (20) 

where  Om  denotes  the  minimum  singular  value  of  Q.  Hence  the  trajectories  starting  outside  S'^  reach  this  set 
in  a  finite  time.  As  in  the  proof  of  Theorem  1,  once  there,  asymptotic  stability  is  guaranteed  by  the  parameter 
dependent  control  Lyapunov  function  '¥{x,  p). 


To  prove  item  2.-  note  that  if  p)  =  (jc,  p)  then  from  the  Hamilton  Jacobi  equation  (9)  we  have  that. 


x{t  +  T)'Qx[t  +  r)  +  min  sup  <  v'Rv+W 

P  [ 


('+n. 


(21) 


Thus  the  constraint  (18)  is  redundant  and  the  proof  follows  immediately  from  Lemma  1. 


□ 


Finally,  before  closing  this  section  we  consider  a  modified  control  law  that  takes  into  account  the  sample 
and  hold  nature  of  receding  horizon  implementations. 


Algorithm  4 

0.-  Data:  a  CLF  'F(j:),  an  invariant  region  Sm  such  that  0  G  a  horizon  T,  a  sampling  interval 

57. 


lin  /  ||m||: 


Egv,<-aW<oj 


1. -  Ifx{t)  G  54/,  m«f(^,P)  =  argmin  i  ||m||:  sup  Lf'¥  +  Lg^*u  +  'Z^Vi  <  - 

“  [  J4<v/<v/ 

2. -  If  x(t)  ^  S'v  then  M'f(jc,  p)  =  t  G  +  57],  where  m*(.)  is  given  by: 

( 7' 

M*  =  argmin  sup  <  /  {x!Qx+u'Ru)dT-\-^[x{T +  t),p{T ■\-t)] 

U  I  J 


(22) 


Object  to: 


0  >  A:(r  +  7  +  T)'(2Jc(t  +  7+T)+min  sup  |«'(t  +  7+T)J?M(r+7  +  T)4-'F|  1-m'(0/?«(0 

“  peJv  I  \{t+T+x)} 


for  all  0  <  T  <  57 


(23) 


Lemma  2  The  control  law  renders  the  origin  a  globally  stable  equilibrium  point  of  (5 ).  Moreover,  it 
coincides  with  the  globally  optimal  control  law  when  ^(x,  p)  =  V(x,  p)  and  57  0. 


Proof:  The  proof,  omitted  for  space  reasons,  follows  along  the  lines  of  the  proof  of  Theorem  2. 
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5  Selecting  suitable  CLFs 


In  principle,  any  of  the  methods  available  in  the  literature  for  finding  CLFs  such  as  feedback  linearization 
and  backstepping  (see  for  instance  [9])  can  be  used  to  find  the  function  'F(.).  Alternatively,  if  a  stabilizing 
linear  static  feedback  control  law  ui^  =  Kx  is  known,  then,  following  [16]  a  suitable  CLF  is  given  by: 

W{x,p)=s\ipf  x'{Q  +  K'RK)xdt 

However,  as  we  show  next,  in  some  cases  of  practical  interest,  specific  families  of  CLFs  are  readily  available 
that  reduce  the  degree  of  suboptimality  incurred  by  the  algorithm. 


5.1  Autonomous  systems 


Consider  the  case  of  autonomous  nonlinear  systems,  i.e.  where  /  =  /(jc)  and  g  =  g(x)  in  (5).  As  we  show 
in  the  sequel,  in  this  case  the  suboptimality  level  incurred  by  the  proposed  algorithm  is  roughly  similar  to 
the  difference  between  the  CLF  'F(., .)  and  the  actual  value  function  F(., .). 


Theorem  3  Let  'F:  R"  -^R+  be  a  positive  definite,  radially  unbounded  function  and  consider  the  following 
optimization  problem: 

fT 

J'v{x,t)  =  min  {xfQx+u'Ru)dx+W[x{T)]  (24) 

subject  to  (5).  Then 


Mx,t)  -  v[x(0]  =  'F[x(r)]  -  v[x(r)]  +  o 
where  e{x,t)  =  y»j/(x,r)  —  V{x)  denotes  the  approximation  error. 


+  0{dt^) 


(25) 


Proof:  By  considering  the  Hamilton  Jacobi  equations  for  Ty  and  V  it  can  be  easily  shown  that  e{t,x)  satisfies 
the  following  equation: 


*  n  f  f  ^  0-1  ,  I  de  j  ,de' 

By  exploiting  the  fact  that  the  optimal  control  law  for  (24)  is  given  by 


equation  (26)  can  be  rewritten  as: 

0 


1  1  fdJ^i 

u^=-^R-8^ 


¥  +  +  ^g'% 


(26) 


(27) 


(28) 
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From  this  last  equation  it  follows  that: 


Ide  __i  ,de' , 
e(T)  =  S  |,(r) 


Expanding  e{t)  in  a  Taylor  series  around  t  =  T  yields: 

e{t)  =  e{T)+e{T)dt  +  0{dt^) 

=  <T)-if,gR-^g'^i'UT)dt+0{dt^) 


(29) 


(30) 

□ 


Corollary  1  Assume  that  'F  is  selected  so  that  =  0.  Then  J'v{x,t)  -  V[x:(r)]  =  'F[x:(r)]  -  V'[x:(r)] 

f  to  the  first  order  in  dt)  along  the  trajectories  of  the  system. 


The  result  above  formalizes  the  intuitively  appealing  fact  that  in  order  to  improve  performance,  the  CLF 
'F(x)  should  be  selected  “close”  to  V.  Next  we  briefly  discuss  an  approach  to  generate  such  functions.  This 
approach  is  motivated  by  the  empirically  observed  success  of  the  State  Dependent  Riccati  Equation  (SDRE) 
method,  briefly  covered  in  the  Appendix. 

From  Lemma  4  in  the  Appendix,  it  follows  that  'F(x:)  =  xlP{x)x,  where  P(x:)  denotes  the  solution  to  the 
SDRE,  is  a  CLF  in  a  neighborhood  of  the  origin.  Moreover,  since  the  control  law  Ugdre  is  locally  stabilizing, 
it  can  be  easily  shown  that  there  exists  To  (possibly  depending  on  the  initial  condition)  such  for  all  T  > 
the  constraints  (18)  are  feasible.  It  follows  that  'F(x:)  =  x!P{x)x  is  a  suitable  choice  for  the  terminal  penalty. 
Moreover  from  the  properties  of  the  SDRE  method  (see  the  Appendix)  it  follows  that  with  this  choice,  the 
control  law  satisfies  all  the  necessary  conditions  for  optimality  as  0[||x:(r  +  r)|p].  Thus,  we  will  expect 
that  Algorithm  3  using  'P(x:)  =x!P{x)x  will  generate  a  nearly  optimal  control  law,  even  when  T  is  relatively 
small.  In  section  8  we  will  show  that  this  is  indeed  the  case  with  two  examples. 


5.2  Linear  Parameter  Varying  Systems 


Consider  now  the  case  where  the  dynamics  (5)  are  linear  in  the  state,  i.e. 

i  =  A[p(t)]x+5[p(r)]M 

This  is  case  of  LPV  systems,  that  has  been  the  object  of  much  attention  in  the  past  few  years  [31],  as  a 
vehicle  to  formalize  the  concept  of  gain  scheduling.  In  this  case  it  can  be  shown  that  a  parameter  dependent 
state  feedback  controller  with  guaranteed  performance  can  be  synthesized  by  solving  the  following  convex 
optimization  problem  (see  [2]  for  details): 


min  TraceZ  (31) 

X(p)>0,Z 
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subject  to: 


-  SSl  +/l(p)X(p)  +X(p)A'(p)  -  S(p)R-'B'(p)  X(py 

^(P)  -Q . 


<0 


-  v/^  +  A(p)X(p)  +X(pM'(p)  -  Bip)R-^B'{p) 

m 


Z  / 

for  all  p  £  IP.  The  corresponding  control  action  is  given  by 

and  the  corresponding  cost  is  bounded  by: 


(32) 


(33) 


•^(•*^0)  Po)  =  sup  f  (^Qx+u'Ru)  dt  <  ’ 


P6?-V^ 


iPo)Xo 


(34) 


While  this  approach  yields  a  stabilizing  controller  with  guaranteed  performance  bounds,  it  is  potentially 
conservative  due  to  the  facts  that  (i)  it  uses  a  quadratic  parameter  dependent  Lyapunov  function  (xfX~^  (p)j^), 
and  (ii)  allows  for  all  possible  combinations  of  the  parameters  and  their  derivatives.  In  the  sequel,  we 
indicate  how  performance  can  be  improved  by  exploiting  the  Receding  Horizon  ideas  presented  in  this 
chapter. 


Lemma  3  Consider  the  case  of  LPV  dynamics.  If  the  terminal  penalty  'P  in  Algorithm  3  is  chosen  as 
^(xiP)  =  ^^“'(p)^:,  where  X  denotes  any  feasible  solution  to  the  set  of  affine  matrix  inequalities  (AMIs) 
(32)  then  the  following  holds: 

1.  The  resulting  control  law  u^^,  (x,  p)  is  globally  stabilizing  for  any  choice  of  the  horizon  T. 

2.  For  any  admissible  parameter  trajectory  p  we  have  that: 

I 

J\v{xo,p)=  [  xlQx+u'Rudt  <x(0yx~\po)x{0)  (35) 

JO 

i.e.  the  proposed  control  law  is  guaranteed  to  perform  no  worse  than  the  AMI  based  control  law  (33), 
in  the  sense  that  both  have  the  same  worse-case  upper  bound. 


Proof:  To  establish  the  first  claim,  note  that  from  the  Euler  Lagrange  conditions  for  optimality,  it  can  be 
easily  shown  that  in  this  case  the  constraint  (18)  is  redundant,  since  it  is  satisfied  by  the  control  that  optimizes 
the  performance  index  (17).  Stability  follows  now  from  Theorem  2. 

To  complete  the  proof,  l&iXAMiMAMhXMf  and  denote  the  trajectory  and  control  corresponding  to  the 
parameter  trajectory  p,  obtained  when  using  the  AMI-based  (33)  and  proposed  control  law  respectively. 
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Then,  from  the  definition  of  it  follows  that: 


-  Sq  {x'^MlQxAMl  +  li'AMI^AMl)dt +XaMi{T)'X  *[p(r)]yAiW/ 


(36) 


From  Schur  complements,  the  set  of  inequalities  (32)  is  equivalent  to 

"p  3v 

-Sv/-^+A[p(0]X(p)+X(p)A[p(0]'+X(p)eX(p)-fl/?-‘5'<0  (37) 

for  all  y,  <  V/  <  v,-  or,  equivalently, 

”p  3X 

'-Jj^'^i-^+AciX  +  XJ^i-\-XQX  +  BR-^B!  <0  (38) 

where  Ac/(p)  =  A[p(r)]-  j5[p(t)]/?“'fi[p(r)]'X(p)  denotes  the  closed-loop  dynamic  matrix.  Pre  and  post- 
multiplying  this  last  equation  by  and  X~^xami  yields  (after  some  algebra): 

’^{x'ami^  ^xami) +x'amiQxami+u'ami^ami  (39) 

Finally,  integrating  this  last  inequality  yields 

/  i^'AMlQ’^AMI-\-u'AMlRiiAMl)dt-\-XAMl{TyX  '[p(7’)}r/iA// <  x(0yX~*  (p(,)x(0)  (40) 

which,  combined  with  (36)  yields  the  desired  result. 


□ 


6  Connections  with  other  approaches 


In  this  ‘section  we  briefly  explore  the  connections  between  the  proposed  controller  and  some  related  ap¬ 
proached  proposed  in  the  past. 

The  basic  idea  of  Algorithm  1,  namely  (i)  to  convert  the  infinite  dimensional  quadratic  regulation  prob¬ 
lem  to  a  finite  dimensional  optimization  by  using  a  penalty  function  to  estimate  the  cost-to-go,  and  (ii) 
to  use  explicit  optimization  to  drive  the  system  to  an  invariant  neighborhood  of  the  origin  and  then  switch 
controllers  was  proposed  in  [25,  26]  for  the  case  of  constrained  LTI  systems  (see  also  [5,  22]  for  later  work 
along  these  lines). 

The  combination  of  dual  mode  control  and  receding  horizon  for  stabilizing  nonlinear  systems  was  pro¬ 
posed  in  [18].  However,  while  this  approach  has  the  ability  to  robustly  stabilize  a  nonlinear  plant,  does  not 
address  the  issue  of  performance.  Indeed,  the  performance  index  used  there  (and  hence  the  control  action) 
does  not  approach  the  optimal  unless  T  From  a  practical  stand-point,  this  implies  that  in  order  to 
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achieve  acceptable  performance,  the  on-line  optimization  must  be  performed  over  a  large  horizon,  which 
may  not  be  feasible  for  moderately  large  plants  or  plants  with  fast  time  constants. 


The  extension  of  the  techniques  proposed  in  [25]  to  nonlinear  plants  was  pursued  in  [16].  Here  the 
infinite  horizon  cost  is  approximated  by  an  expression  of  the  form  (11),  where  the  terminal  penalty  function 
is  obtained  assuming  that  a  stabilizing  control  law  of  the  form  u  =  -Kx  is  available  and  will  be  used  after  the 
horizon  T.  Alternatively,  the  penalty  function  can  be  obtained  by  solving  the  Ricatti  equation  corresponding 
to  the  linearization  of  the  dynamics  around  the  equilibrium  point  x  =  0  (see  also  [4]).  Thus,  this  approach 
can  be  viewed  as  a  special  case  of  Algorithm  1  for  a  particular  choice  of  (local)  CLF. 


The  combination  of  Receding  Horizon  and  CLF  techniques  has  been  proposed  independently  in  [28, 29] 
and  [20].  A  different  between  these  approaches  is  that  the  latter  does  not  include  a  terminal  penalty  in  the 
performance  index.  Rather,  [20]  optimizes  a  performance  index  of  the  form: 


t+T 


t 


(41) 


subject  to 


LfV{x)  +  LgV{x)u  <  —Ba{x) 

v[x(r+r)]  <  KMt+r)] 


(42) 


where  V(.)  is  a  CLF  and  X(,  is  the  trajectory  corresponding  to  the  pointwise  minimum  norm  control  that 
renders  V  <  0.  This  approach  is  guaranteed  to  stabilize  the  system  for  any  horizon  T  such  that  the  constraints 
(42)  are  feasible.  However,  it  may  do  so  at  the  expense  of  performance.  Note  that  contrary  to  (11),  (41) 
does  not  approximate  the  original  cost,  even  if  T  is  taken  large  or  V  coincides  with  the  actual  value  function 
for  the  problem.  Thus,  the  corresponding  control  action,  while  stabilizing,  is  not  necessarily  close  to  the 
optimal. 


Finally,  we  close  this  section  by  showing  that  in  the  limit  as  T  ^  0,  the  control  law  obtained  from 
Algorithm  3  reduces  to  the  inverse  optimal  controller  proposed  by  Freeman  and  Kokotovic  [8].  To  this 
effect,  note  that  if  7  0  in  (17)  then  is  given  by  the  solution  to  the  following  optimization  problem: 


My  =  aigmin  {u'Ru  + 

U 

(43) 

subject  to: 

i 

\|/o  +  Vl«  =  0 

(44) 

where: 

\\fo  =  L/T[x(r)]+x(/)'(2x(0+  sup  SSp/  +  a[x(0] 

yi<(>.<v/i=i 

Vi  = 

(45) 

and  where  -a(x:)  is  the  desired  negativity  margin  [8].  The  solution  to  this  optimization  problem  is  given  by: 


vitoR~Vi 

¥i^"Vi 


(46) 


This  is  precisely  the  inverse  optimal  controller  obtained  in  [8] 
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7  Incorporating  constraints 


Next,  we  briefly  discuss  how  to  incorporate  constraints  into  the  formalism,  proceeding  in  the  spirit  of  [25, 
27]. 


Assume  for  instance  that  the  control  action  is  constrained  to  belong  to  a  given  compact,  convex,  possibly 
parameter  dependent  set  u  £  ‘ii(p).  Let  denote  the  unconstrained  value  function  for  the  problem  in  a 

region  S,  and  consider  the  set  Zly  =  |jc;m  =  6  “^(p)  for  P  £  i-O-  the  set  of  states  where 

the  unconstrained  control  law  satisfies  the  control  constraints.  Finally,  denote  by  Sy  the  largest  invariant 
set  contained  in  tly.  Then,  Algorithm  1  can  be  applied  to  the  problem  by  simply  modifying  the  explicit 
optimization  (11)  to  take  the  constraints  into  account  and  using  the  set  Sy  as  Sy  Clearly,  this  modified 
control  law  has  the  same  properties  as  Algorithm  1,  i.e.  if  the  optimization  is  feasible,  then  it  stabilizes 
the  system  and  yields  the  optimal  control  action.  Similarly,  the  use  of  a  fixed  optimization  horizon  T  and 
an  approximation  '^(x.p)  can'be  taken  into  account  by  selecting  T  to  be  a  constrained  control  Lyapunov 
function  in  the  sense  that: 


min  sup 


dx 


[/(■*!  P)+g(-r,P)M]  + 


<  <  Oi  Vx6  5v 


(47) 


and  modifying  the  algorithm  so  that  the  minimization  is  taken  over  u  £  “[/(p)  rather  than  over  R"".  Clearly, 
this  algorithm  stabilizes  the  system  in  the  region  where  this  can  be  accomplished  with  bounded  controls. 
Finally,  state  constraints  can  also  be  incorporated  into  the  formalism  by  rendering  suitable  sets  invariant, 
proceeding  as  in  [27]. 


8  Illustrative  Examples 


This  section  illustrates  our  results  with  several  examples.  The  first  one  is  a  simple  academic  example  that  can 
be  solved  analytically.  Thus  it  can  be  used  to  explicitly  analyze  the  source  of  performance  degradation  when 
using  feedback  linearization  or  the  SDRE  method,  and  to  show,  the  advantages  of  the  proposed  approach. 
The  second  example,  a  realistic  problem  arising  in  the  context  of  control  of  thrust  vectored  aircrafts,  was 
used  for  benchmarking  several  nonlinear  design  methods  in  [7].  Finally,  the  third  example  illustrates  the 
advantages  of  the  proposed  method  for  the  case  of  LPV  systems. 

Example  1:  Consider  the  following  regulation  problem: 


min 

U 


J  = 


subject  to 

Xl  =  X2 

X2  =  + 

It  can  be  shown  that  the  optimal  control  law  is  given  by: 


Uopt  =  -X2 


(48) 


(49) 


(50) 
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Figure  2:  State  trajectories  for  example  1 


Figure  3:  Control  effort  and  cost  for  example  1 


with  the  corresponding  optimal  storage  function: 

V(x:)  =jc^  . 


(51) 


A  feedback  linearization  design  selected  so  that  the  closed-loop  system  has  the  same  storage  function  as 
||x||  ^  0  yields  the  following  controller  and  Lyapunov  function: 


“ft,  =  (1—e  ^‘)xi—X2e  ( 1  +  0.5 * X2) 

VfL  = 


(52) 


Note  that  ufl  =  Uopt  only  for  small  values  of  xi  and  X2.  Consider  now  the  following  state-dependent 
coefficient  (SDC)  parametrization: 


A  = 


0  1 


B: 


0 


It  can  be  shown  that  the  solution  to  the  corresponding  SDRE  is  given  by: 


P{x)  = 


e*'  0 

0  1 


p{x) 


(53) 


(54) 
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where 


p{x) 


2^  +  \/‘+(^) 


with  associated  control  action: 


^sdre  —  ^^2 


Finally,  it  can  also  be  shown  that 

xf  F(x)j:  =  V  (x) 

Thus  xfP(x)x  gives  a  good  estimate  of  V (jc)  and  u^dre  —  Uopt  only  when  (^)  <  1  • 


(55) 


(56) 

(57) 


Table  2  shows  the  different  costs  starting  from  the  initial  condition  j[:(0)  =  [-2  2]  for  several  con¬ 
trollers,  with  the  corresponding  trajectories  shown  in  Figures  2  and  3.  The  last  two  entries  of  the  table 
correspond  to  the  the  proposed  controller  using  an  horizon  T  =  lsec  and  as  estimates  of  the  value  function 
'F  =  Vfi  and  'F  =  xfP{x)x  respectively.  Note  that  in  this  case  performance  of  both  controllers  is  virtually 
identical  to  optimal. 


method 

cost 

optimal 

33.56 

FL 

95.11 

SDRE 

143.0 

RH+FL 

34.24 

RH+SDRE 

33.7 

Table  2:  Comparison  of  different  methods  for  Example  1 


Example  2:  Consider  again  the  simplified  model  of  the  thrust  vectored  aircraft  used  in  the  introduction. 
Table  3  shows  the  cost  corresponding  to  the  initial  condition  ^(0)  =  [0  0  0  12.5  0  0],  obtained 

using  different  controllers.  The  two  lowest  entries  correspond  to  the  proposed  method  using  T  —  Isec  and 
Ts  =  0.55gc  and  terminal  penalties  derived  from  Jacobian  Linearization  and  the  SDRE  methods  respectively. 
Note  that  the  latter  virtually  achieves  optimal  performance,  while  the  former  is  only  2%  suboptimal.  This 
behavior  can  be  explained  by  looking  at  Figure  4  that  shows  the  different  portions  of  the  cost  as  a  function 
of  the  horizon,  starting  from  the  initial  condition  ^(0).  These  plots  show  that  while  'F(x)  =  xfP{x)x  gives 
initiallj'  a  very  poor  estimate  of  the  cost-to-go,  the  combination  of 'F(x)  and  the  explicit  integral  in  (17) 
give  a  very  good  estimate  if  T  is  chosen  >  Isec.  It  is  worth  mentioning  that  a  conventional  receding  horizon 
controller  (i.e.  one  obtained  by  setting  'F  =  0  in  (17))  with  the  same  choice  of  horizon  and  sampling  time 
fails  to  stabilize  the  system. 


Example  3:  Consider  an  LPV  system  with  the  following  state  space  realization: 


A 


0  1 
1  0.5P-1.5 


52  =  [0  1]', 


Cl 


-1 

0 


0,2  =  [0  1]' 


(58) 


P  =  {p:0<  p  <  1};  y= -2,v  =  2 
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method 

cost 

exact 

1115 

LQR  [7] 

1.1x10^ 

CLF  [7] 

2.53  X  10^ 

LPV  [7] 

1833 

SDRE 

1640 

RH+JL 

1142(T=1) 
1321  (T=0.4) 

RH+SDRE 

1117  (T=l) 
1310(T=0.4) 

Table  3:  Comparison  of  different  methods  for  Example  2 


Dfan:  x(0)=[0  0  0  12.5  0  0],  Th=1,  Ts=0.5 


Figure  4;  The  terms  of  the  cost  as  function  of  the  horizon 


It  can  be  easily  verified  that  the  following  matrix  function  satisfies  the  AMIs  (32): 
X(p)  =  Xo+X\p  +  X2P^ 


0.2210 

-0.3505 

-0.3505' 
1.1272  J 

,X,= 

■ -0.0239 
0.0924 

0.0924  ■ 
-0.3577_ 

(59) 

0.0243 

-0.0683' 

-0.0683 

0.2180 

for  all  p  6  !Fv  Figure  5‘^  compares  the  trajectories  starting  from  the  initial  condition  [0  2]'  for  the  AMI- 
based  {xupv,X2ipv,  uipv)  and  the  proposed  controller  respectively.  The  latter  was  implemented  using  7  =  2  as 
horizon  and  'F  =  xfX~\p)x.  For  the  specific  parameter  history  shown  there,  the  receding  horizon  controller 
yields  J  =  6.91  versus  J  =  8.30  for  the  control  law  (33),  a  performance  improvement  of  roughly  20%. 
Similar  results  were  obtained  for  other  initial  conditions  and  parameter  trajectories. 

^In  this  figure  the  parameter  was  normalized  to  p„  =  0.5  ♦  p  —  1.5. 
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state  history:  x1 


time  (sec.) 
Control  action 


State  history:  x2 


Figure  5:  state,  control  and  (normalized)  parameter  trajectories  for  the  Example 


9  Conclusions 


In  contrast  with  the  case  of  linear  plants,  tools  for  simultaneously  addressing  performance  and  stability  of 
nonlinear  systems  have  emerged  relatively  recently.  Recent  counterexamples  [7]  illustrated  the  fact  that 
while  several  commonly  used  techniques  can  successfully  stabilize  nonlinear  systems,  the  resulting  closed- 
loop  performance  varies  widely.  Moreover,  these  performance  differences  are  problem  dependent,  with 
performance  of  a  given  method  ranging  from  (near)  optimal  to  very  poor. 

In  this  chapter  we  have  proposed  a  new  suboptimal  regulator  for  control  affine  parameter  dependent  non¬ 
linear  systems,  based  upon  the  combination  of  receding  horizon  and  control  Lyapunov  functions  techniques. 
The  main  result  of  the  chapter  shows  that  under  certain  relatively  mild  conditions,  essentially  equivalent  to 
the  existence  of  a  Control  Lyapunov  Function,  this  regulator  renders  the  origin  a  globally  asymptotically 
stable  equilibrium  point.  Additional  results  show  that  for  some  readily  available  choices  for  the  CLF  V|/(x,  p) 
render  the  proposed  controller  near  optimal  for  some  cases  of  practical  interest.  These  results  were  illus¬ 
trated  with  a  number  of  examples,  where  the  proposed  controller  outperformed  several  other  commonly 
used  techniques. 

An  issue  that  was  not  addressed  here  is  that  of  the  computational  complexity  associated  with  solving 
the  nonlinear  optimization  problem  (17).  Following  [7]  this  complexity  could  be  reduced  by  exploiting 
differential  flatness  to  perform  the  optimization  in  flat  space.  Additional  research  being  pursued  includes 
the  extension  of  the  framework  to  the  output  feedback  case  and  to  handle  model  uncertainty. 
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A  The  SDRE  approach  to  nonlinear  regulation 


In  this  section  we  briefly  cover  the  details  of  the  SDRE  approach  developed  by  Cloutier  and  coworkers 
[6,  19].  The  main  idea  of  the  method  is  to  recast  the  the  nonlinear  system  (5)  into  a  State  Dependent 
Coefficient  (SCD)  linear-like  form: 

x  =  A  (x)x  +B{x)u  (60) 
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and  to  solve  pointwise  along  the  trajectory  the  corresponding  algebraic  Riccati  equation: 


A'(A:)P(j:)  +  P(j;)i4(j:)-P(x)fi(jc)R  *(x)B'(-c)P(j:)  +  2(a:)  =  0  (61) 

The  suboptimal  control  law  is  given  by  Usdre  —  —  ^R~^ix)B'P{x)x  where  P(x)  is  the  positive  definite  (point- 
wise  stabilizing)  solution  of  (61).  In  the  sequel  we  briefly  review  the  properties  of  this  control  law.  The 
corresponding  proofs  can  be  found  in  the  appropriate  references. 


Lemma  4  ([6, 19])  Assume  that  Q{x)  =  C'{x)C{x)  and  that  there  exists  a  neighborhood  of  the  origin 
where  the  pairs  {A(jt),  5(a:)}  and  {A(;c),C(.r)}  are  pointwise  stabilizable  and  detectable  respectively  and  all 
the  matrix  functions  involved  are  C^.  Then  the  control  law  u^dre  renders  the  origin  a  locally  asymptotically 
stable  equilibrium  point  of  the  closed-loop  system. 


Lemma  5  ([6, 19])  The  SDRE  control  law  and  its  associated  state  and  co-state  trajectories  satisfy  the 
following  necessary  condition  for  optimality:  ^  =  0  where  H  —  x!Q{x)x-]r  u'R(x)u  +  X'  [/(j:)  -1-5(jc)m]  is 
the  Hamiltonian  of  the  system  and  where  X  denotes  the  co-states, 


Lemma  6  ([6, 19])  Assume  that  the  parametrization  ( 60)  is  stabilizable  and  all  the  matrices  involved  along 
with  their  gradients  are  bounded  in  a  neighborhood  Q  of  the  origin.  Then  the  SDRE  control  law  and  its 
associated  state  and  co-state  trajectories  asymptotically  satisfy  at  a  quadratic  rate^  following  necessary 
condition  for  optimality:  X  =  in  the  sense  that 

\\^+^\\<xfUx 


for  some  constant  matrix  U  >0  and  all  a:  £  Q. 


Lemma  7  ([12])  Let  P[x)  denote  a  solution  to  the  SDRE  (61).  If  there  exists  a  positive  definite  function 
V  (x)  such  that  =  P(x)jc  then  Usdre  is  the  globally  optimal  control  law^. 


^i.e.  ||X+^||-+0as  0(||j:|p)as.x-iO. 

necessary  and  sufficient  condition  for  this  to  hold  is  that  the  Jacobian  matrix  ^  [P(jc)x:]  is  symmetric  (see  for  instance  [13], 
section  3.1).  In  this  case  V (x)  can  be  computed  as  K  (j:)  =  y'P(y)  ■  dy,  where  this  line  integral  is  independent  of  the  path. 
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